1 00:00:00,000 --> 00:00:07,930 2 00:00:07,930 --> 00:00:13,530 OK, here we go with, quiz review for the third quiz that's 3 00:00:13,530 --> 00:00:15,910 coming on Friday. 4 00:00:15,910 --> 00:00:19,040 So, one key point is that the quiz 5 00:00:19,040 --> 00:00:25,120 covers through chapter six. 6 00:00:25,120 --> 00:00:27,580 Chapter seven on linear transformations 7 00:00:27,580 --> 00:00:31,930 will appear on the final exam, but not on the quiz. 8 00:00:31,930 --> 00:00:35,540 So I won't review linear transformations today, 9 00:00:35,540 --> 00:00:39,720 but they'll come into the full course review on the very 10 00:00:39,720 --> 00:00:41,350 last lecture. 11 00:00:41,350 --> 00:00:44,120 So today, I'm reviewing chapter six, 12 00:00:44,120 --> 00:00:46,330 and I'm going to take some old exams, 13 00:00:46,330 --> 00:00:48,990 and I'm always ready to answer questions. 14 00:00:48,990 --> 00:00:54,360 And I thought, kind of help our memories if I write down 15 00:00:54,360 --> 00:01:00,350 the main topics in chapter six. 16 00:01:00,350 --> 00:01:04,019 So, already, on the previous quiz, 17 00:01:04,019 --> 00:01:07,520 we knew how to find eigenvalues and eigenvectors. 18 00:01:07,520 --> 00:01:13,480 Well, we knew how to find them by that determinant of A minus 19 00:01:13,480 --> 00:01:15,390 lambda I equals zero. 20 00:01:15,390 --> 00:01:17,280 But, of course, there could be shortcuts. 21 00:01:17,280 --> 00:01:20,030 There could be, like, useful information 22 00:01:20,030 --> 00:01:27,190 about the eigenvalues that we can speed things up with. 23 00:01:27,190 --> 00:01:27,830 OK. 24 00:01:27,830 --> 00:01:32,300 Then, the new stuff starts out with a differential equation, 25 00:01:32,300 --> 00:01:34,680 so I'll do a problem. 26 00:01:34,680 --> 00:01:36,810 I'll do a differential equation problem first. 27 00:01:36,810 --> 00:01:39,970 28 00:01:39,970 --> 00:01:43,010 What's special about symmetric matrices? 29 00:01:43,010 --> 00:01:46,060 Can we just say that in words? 30 00:01:46,060 --> 00:01:48,170 I'd better write it down, though. 31 00:01:48,170 --> 00:01:50,960 What's special about symmetric matrices? 32 00:01:50,960 --> 00:01:56,970 Their eigenvalues are real. 33 00:01:56,970 --> 00:02:00,970 The eigenvalues of a symmetric matrix always come out real, 34 00:02:00,970 --> 00:02:04,800 and there always are enough eigenvectors. 35 00:02:04,800 --> 00:02:06,930 Even if there are repeated eigenvalues, 36 00:02:06,930 --> 00:02:09,910 there are enough eigenvectors, and we 37 00:02:09,910 --> 00:02:13,180 can choose those eigenvectors to be orthogonal. 38 00:02:13,180 --> 00:02:16,480 So if A equals A transposed, the big fact 39 00:02:16,480 --> 00:02:22,620 will be that we can diagonalize it, 40 00:02:22,620 --> 00:02:28,550 and those eigenvector matrix, with the eigenvectors 41 00:02:28,550 --> 00:02:31,470 in the column, can be an orthogonal matrix. 42 00:02:31,470 --> 00:02:36,600 So we get a Q lambda Q transpose. 43 00:02:36,600 --> 00:02:43,010 That, in three symbols, expresses a wonderful fact, 44 00:02:43,010 --> 00:02:46,250 a fundamental fact for symmetric matrices. 45 00:02:46,250 --> 00:02:46,960 OK. 46 00:02:46,960 --> 00:02:49,740 Then, we went beyond that fact to ask 47 00:02:49,740 --> 00:02:52,570 about positive definite matrices, when 48 00:02:52,570 --> 00:02:54,370 the eigenvalues were positive. 49 00:02:54,370 --> 00:02:55,970 I'll do an example of that. 50 00:02:55,970 --> 00:02:59,290 51 00:02:59,290 --> 00:03:01,580 Now we've left symmetry. 52 00:03:01,580 --> 00:03:05,000 Similar matrices are any square matrices, 53 00:03:05,000 --> 00:03:10,730 but two matrices are similar if they're related that way. 54 00:03:10,730 --> 00:03:14,590 And what's the key point about similar matrices? 55 00:03:14,590 --> 00:03:16,770 Somehow, those matrices are representing 56 00:03:16,770 --> 00:03:19,410 the same thing in different basis, 57 00:03:19,410 --> 00:03:21,980 in chapter seven language. 58 00:03:21,980 --> 00:03:29,220 In chapter six language, what's up with these similar matrices? 59 00:03:29,220 --> 00:03:31,860 What's the key fact, the key positive fact 60 00:03:31,860 --> 00:03:33,570 about similar matrices? 61 00:03:33,570 --> 00:03:37,130 They have the same eigenvalues. 62 00:03:37,130 --> 00:03:39,070 Same eigenvalues. 63 00:03:39,070 --> 00:03:42,460 So if one of them grows, the other one grows. 64 00:03:42,460 --> 00:03:50,380 If one of them decays to zero, the other one decays to zero. 65 00:03:50,380 --> 00:03:53,050 Powers of A will look like powers of B, 66 00:03:53,050 --> 00:03:55,440 because powers of A and powers of B 67 00:03:55,440 --> 00:03:59,160 only differ by an M inverse and an M way on the 68 00:03:59,160 --> 00:04:00,040 outside. 69 00:04:00,040 --> 00:04:05,250 So if these are similar, then B to the k-th power 70 00:04:05,250 --> 00:04:09,160 is M inverse A to the k-th power M. 71 00:04:09,160 --> 00:04:12,410 And that's why I say, eh, this M, it 72 00:04:12,410 --> 00:04:14,480 does change the eigenvectors, but it 73 00:04:14,480 --> 00:04:16,089 doesn't change the eigenvalues. 74 00:04:16,089 --> 00:04:21,209 So same lambdas. 75 00:04:21,209 --> 00:04:24,510 And then, finally, I've got to review the point 76 00:04:24,510 --> 00:04:30,000 about the SVD, the Singular Value Decomposition. 77 00:04:30,000 --> 00:04:30,500 OK. 78 00:04:30,500 --> 00:04:33,900 So that's what this quiz has got to cover, 79 00:04:33,900 --> 00:04:37,720 and now I'll just take problems from earlier exams, 80 00:04:37,720 --> 00:04:40,460 starting with a differential equation. 81 00:04:40,460 --> 00:04:41,250 OK. 82 00:04:41,250 --> 00:04:43,360 And always ready for questions. 83 00:04:43,360 --> 00:04:46,640 So here is an exam from about the year 84 00:04:46,640 --> 00:04:53,290 zero, and it has a three by three. 85 00:04:53,290 --> 00:04:54,502 So that was -- 86 00:04:54,502 --> 00:04:57,280 87 00:04:57,280 --> 00:04:59,720 but it's a pretty special-looking matrix, 88 00:04:59,720 --> 00:05:04,190 it's got zeroes on the diagonal, it's got minus ones above, 89 00:05:04,190 --> 00:05:08,360 and it's got plus ones like that. 90 00:05:08,360 --> 00:05:12,160 So that's the matrix A. 91 00:05:12,160 --> 00:05:12,880 OK. 92 00:05:12,880 --> 00:05:16,750 Step one is, well, I want to solve that 93 00:05:16,750 --> 00:05:17,700 equation. 94 00:05:17,700 --> 00:05:19,970 I want to find the general solution. 95 00:05:19,970 --> 00:05:22,790 I haven't given you a u(0) here, so I'm 96 00:05:22,790 --> 00:05:24,900 looking for the general solution, 97 00:05:24,900 --> 00:05:28,280 so now what's the form of the general solution? 98 00:05:28,280 --> 00:05:30,520 With three arbitrary constants going 99 00:05:30,520 --> 00:05:33,940 to be inside it, because those will be used 100 00:05:33,940 --> 00:05:35,780 to match the initial condition. 101 00:05:35,780 --> 00:05:39,580 So the general form is u at time t 102 00:05:39,580 --> 00:05:45,500 is some multiple of the first special solution. 103 00:05:45,500 --> 00:05:50,080 The first special solution will be growing like the eigenvalue, 104 00:05:50,080 --> 00:05:51,920 and it's the eigenvector. 105 00:05:51,920 --> 00:05:56,400 So that's a pure exponential solution, just staying 106 00:05:56,400 --> 00:05:58,580 with that eigenvector. 107 00:05:58,580 --> 00:06:01,770 Of course, I haven't found, yet, the eigenvalues 108 00:06:01,770 --> 00:06:02,580 and eigenvectors. 109 00:06:02,580 --> 00:06:04,900 That's, normally, the first job. 110 00:06:04,900 --> 00:06:08,880 Now, there will be second one, growing like e 111 00:06:08,880 --> 00:06:13,160 to the lambda two, and a third one growing like e 112 00:06:13,160 --> 00:06:16,450 to the lambda three. 113 00:06:16,450 --> 00:06:18,880 So we're all done -- 114 00:06:18,880 --> 00:06:20,860 well, we haven't done anything yet, 115 00:06:20,860 --> 00:06:22,100 actually. 116 00:06:22,100 --> 00:06:25,930 I've got to find the eigenvalues and eigenvectors, 117 00:06:25,930 --> 00:06:30,720 and then I would match u(0) by choosing the right three 118 00:06:30,720 --> 00:06:31,420 constants. 119 00:06:31,420 --> 00:06:31,950 OK. 120 00:06:31,950 --> 00:06:35,030 So now I ask -- ask you about the eigenvalues 121 00:06:35,030 --> 00:06:38,790 and eigenvectors, and you look at this matrix and what do you 122 00:06:38,790 --> 00:06:41,390 see in that matrix? 123 00:06:41,390 --> 00:06:47,740 Um, well, I guess we might ask ourselves right away, 124 00:06:47,740 --> 00:06:50,760 is it singular? 125 00:06:50,760 --> 00:06:51,810 Is it singular? 126 00:06:51,810 --> 00:06:54,040 Because, if so, then we really have a head start, 127 00:06:54,040 --> 00:06:56,810 we know one of the eigenvalues is zero. 128 00:06:56,810 --> 00:06:57,990 Is that matrix singular? 129 00:06:57,990 --> 00:07:01,660 130 00:07:01,660 --> 00:07:04,280 Eh, I don't know, do you take the determinant to 131 00:07:04,280 --> 00:07:05,110 find out? 132 00:07:05,110 --> 00:07:08,530 Or maybe you look at the first row and third row 133 00:07:08,530 --> 00:07:10,270 and say, hey, the first row and third row 134 00:07:10,270 --> 00:07:15,180 are just opposite signs, they're linear-dependent? 135 00:07:15,180 --> 00:07:17,930 The first column and third column are dependent -- it's 136 00:07:17,930 --> 00:07:19,150 singular. 137 00:07:19,150 --> 00:07:22,610 So one eigenvalue is zero. 138 00:07:22,610 --> 00:07:24,520 Let's make that lambda one. 139 00:07:24,520 --> 00:07:26,150 Lambda one, then, will be zero. 140 00:07:26,150 --> 00:07:26,850 OK. 141 00:07:26,850 --> 00:07:30,000 Now we've got a couple of other eigenvalues to find, 142 00:07:30,000 --> 00:07:33,430 and, I suppose the simplest way is 143 00:07:33,430 --> 00:07:36,940 to look at A minus lambda I So let 144 00:07:36,940 --> 00:07:43,720 me just put minus lambda in here, minus ones above, 145 00:07:43,720 --> 00:07:45,520 ones below. 146 00:07:45,520 --> 00:07:51,430 But, actually, before I do it, that matrix 147 00:07:51,430 --> 00:07:53,910 is not symmetric, for sure, right? 148 00:07:53,910 --> 00:07:57,220 In fact, it's the very opposite of symmetric. 149 00:07:57,220 --> 00:08:02,420 That matrix A transpose, how is A transpose connected to A? 150 00:08:02,420 --> 00:08:04,020 It's negative A. 151 00:08:04,020 --> 00:08:07,890 It's an anti-symmetric matrix, skew-symmetric matrix. 152 00:08:07,890 --> 00:08:11,230 And we've met, maybe, a two-by-two example 153 00:08:11,230 --> 00:08:14,370 of skew-symmetric matrices, and let 154 00:08:14,370 --> 00:08:17,687 me just say, what's the deal with their eigenvalues? 155 00:08:17,687 --> 00:08:18,645 They're pure imaginary. 156 00:08:18,645 --> 00:08:21,400 157 00:08:21,400 --> 00:08:23,440 They'll be on the imaginary axis, 158 00:08:23,440 --> 00:08:26,710 there be some multiple of I if it's 159 00:08:26,710 --> 00:08:29,090 an anti-symmetric, skew-symmetric matrix. 160 00:08:29,090 --> 00:08:31,360 So I'm looking for multiples of I, 161 00:08:31,360 --> 00:08:33,730 and of course, that's zero times I, 162 00:08:33,730 --> 00:08:37,760 that's on the imaginary axis, but maybe I just do it out, 163 00:08:37,760 --> 00:08:38,350 here. 164 00:08:38,350 --> 00:08:39,740 Lambda cubed. 165 00:08:39,740 --> 00:08:42,600 well, maybe that's minus lambda cubed, 166 00:08:42,600 --> 00:08:44,480 and then a zero and a zero. 167 00:08:44,480 --> 00:08:47,870 Zero, and then maybe I have a plus a lambda, 168 00:08:47,870 --> 00:08:52,610 and another plus lambda, but those go with a minus sign. 169 00:08:52,610 --> 00:08:55,080 Am I getting minus two lambda equals zero? 170 00:08:55,080 --> 00:08:58,640 171 00:08:58,640 --> 00:08:59,940 So. 172 00:08:59,940 --> 00:09:04,810 So I'm solving lambda cube plus two lambda equals zero. 173 00:09:04,810 --> 00:09:09,070 So one root factors out lambda, and the the rest 174 00:09:09,070 --> 00:09:11,571 is lambda squared plus two. 175 00:09:11,571 --> 00:09:12,070 OK. 176 00:09:12,070 --> 00:09:14,310 This is going the way we expect, right? 177 00:09:14,310 --> 00:09:22,360 Because this gives the root lambda equals zero, and gives 178 00:09:22,360 --> 00:09:27,630 the other two roots, which are lambda equal what? 179 00:09:27,630 --> 00:09:30,200 The solutions of when is lambda squared 180 00:09:30,200 --> 00:09:36,510 plus two equals zero then the eigenvalues those guys, what 181 00:09:36,510 --> 00:09:38,610 are they? 182 00:09:38,610 --> 00:09:41,260 They're a multiple of i, they're just square root of two 183 00:09:41,260 --> 00:09:42,150 i. 184 00:09:42,150 --> 00:09:45,010 When I set this equals to zero, I 185 00:09:45,010 --> 00:09:48,800 have lambda squared equal to minus two, right? 186 00:09:48,800 --> 00:09:50,710 To make that zero? 187 00:09:50,710 --> 00:09:54,180 And the roots are square root of two i 188 00:09:54,180 --> 00:09:58,060 and minus the square root of two i. 189 00:09:58,060 --> 00:09:59,540 So now I know what those are. 190 00:09:59,540 --> 00:10:01,070 I'll put those in, now. 191 00:10:01,070 --> 00:10:03,840 Either the zero t is just a one. 192 00:10:03,840 --> 00:10:06,840 That's just a one. 193 00:10:06,840 --> 00:10:13,700 This is square root of two I and this is minus 194 00:10:13,700 --> 00:10:18,280 square root of two I. 195 00:10:18,280 --> 00:10:22,590 So, is the solution decaying to zero? 196 00:10:22,590 --> 00:10:25,210 Is this a completely stable problem 197 00:10:25,210 --> 00:10:28,630 where the solution is going to zero? 198 00:10:28,630 --> 00:10:31,200 No. 199 00:10:31,200 --> 00:10:34,620 In fact, all these things are staying the same size. 200 00:10:34,620 --> 00:10:39,290 This thing is getting multiplied by this number. 201 00:10:39,290 --> 00:10:46,880 e to the I something t, that's a number that has magnitude one, 202 00:10:46,880 --> 00:10:50,160 and sort of wanders around the unit circle. 203 00:10:50,160 --> 00:10:51,660 Same for this. 204 00:10:51,660 --> 00:10:55,900 So that the solution doesn't blow up, and it doesn't go to 205 00:10:55,900 --> 00:10:56,520 zero. 206 00:10:56,520 --> 00:10:57,410 OK. 207 00:10:57,410 --> 00:10:59,670 And to find out what it actually is, 208 00:10:59,670 --> 00:11:02,370 we would have to plug in initial conditions. 209 00:11:02,370 --> 00:11:04,410 But actually, the next question I ask 210 00:11:04,410 --> 00:11:12,280 is, when does the solution return to its initial value? 211 00:11:12,280 --> 00:11:15,910 I won't even say what's the initial value. 212 00:11:15,910 --> 00:11:22,540 This is a case in which I think this solution is 213 00:11:22,540 --> 00:11:25,350 periodic after. 214 00:11:25,350 --> 00:11:31,820 At t equals zero, it starts with c1, c2, and c3, 215 00:11:31,820 --> 00:11:36,500 and then at some value of t, it comes back to that. 216 00:11:36,500 --> 00:11:38,440 So that's a very special question, 217 00:11:38,440 --> 00:11:40,280 Well, let's just take three seconds, 218 00:11:40,280 --> 00:11:44,040 because that special question isn't likely to be on the quiz. 219 00:11:44,040 --> 00:11:49,910 But it comes back to the start, when? 220 00:11:49,910 --> 00:11:55,230 Well, whenever we have e to the two pi i, that's one, 221 00:11:55,230 --> 00:11:56,450 and we've come back again. 222 00:11:56,450 --> 00:11:58,700 So it comes back to the start. 223 00:11:58,700 --> 00:12:07,740 It's periodic, when this square root of two i -- 224 00:12:07,740 --> 00:12:10,930 shall I call it capital T, for the period? 225 00:12:10,930 --> 00:12:17,270 For that particular T, if that equals two pi i, then e 226 00:12:17,270 --> 00:12:21,090 to this thing is one, and we've come around again. 227 00:12:21,090 --> 00:12:26,250 So the period is T is determined here, cancel the i-s, 228 00:12:26,250 --> 00:12:30,970 and T is pi times the square root of two. 229 00:12:30,970 --> 00:12:32,270 So that's pretty neat. 230 00:12:32,270 --> 00:12:35,610 We get all the information about all solutions, 231 00:12:35,610 --> 00:12:39,230 we haven't fixed on only one particular solution, 232 00:12:39,230 --> 00:12:41,450 but it comes around again. 233 00:12:41,450 --> 00:12:43,730 So this was probably my first chance 234 00:12:43,730 --> 00:12:46,270 to say something about the whole family 235 00:12:46,270 --> 00:12:49,760 of anti-symmetric, skew-symmetric matrices. 236 00:12:49,760 --> 00:12:50,380 OK. 237 00:12:50,380 --> 00:12:56,880 And then, finally, I asked, take two eigenvectors (again, 238 00:12:56,880 --> 00:12:59,450 I haven't computed the eigenvectors) 239 00:12:59,450 --> 00:13:02,690 and it turns out they're orthogonal. 240 00:13:02,690 --> 00:13:03,500 They're orthogonal. 241 00:13:03,500 --> 00:13:06,500 The eigenvectors of a symmetric matrix, 242 00:13:06,500 --> 00:13:10,250 or a skew-symmetric matrix, are always orthogonal. 243 00:13:10,250 --> 00:13:13,200 244 00:13:13,200 --> 00:13:18,020 I guess may conscience makes me tell you, 245 00:13:18,020 --> 00:13:23,950 what are all the matrices that have orthogonal eigenvectors? 246 00:13:23,950 --> 00:13:27,040 And symmetric is the most important class, 247 00:13:27,040 --> 00:13:28,640 so that's the one we've spoken about. 248 00:13:28,640 --> 00:13:32,800 But let me just put that little fact down, here. 249 00:13:32,800 --> 00:13:36,661 Orthogonal x-s. 250 00:13:36,661 --> 00:13:37,202 eigenvectors. 251 00:13:37,202 --> 00:13:41,430 252 00:13:41,430 --> 00:13:44,200 A matrix has orthogonal eigenvectors, 253 00:13:44,200 --> 00:13:47,430 the exact condition -- it's quite beautiful that I can tell 254 00:13:47,430 --> 00:13:49,340 you exactly when that happens. 255 00:13:49,340 --> 00:13:55,290 It happens when A times A transpose equals A transpose 256 00:13:55,290 --> 00:14:00,290 times A. Any time that's the condition 257 00:14:00,290 --> 00:14:03,700 for orthogonal eigenvectors. 258 00:14:03,700 --> 00:14:09,040 And because we're interested in special families of vectors, 259 00:14:09,040 --> 00:14:12,690 tell me some special families that fit. 260 00:14:12,690 --> 00:14:15,410 This is the whole requirement. 261 00:14:15,410 --> 00:14:21,120 That's a pretty special requirement most matrices have. 262 00:14:21,120 --> 00:14:23,070 So the average three-by-three matrix 263 00:14:23,070 --> 00:14:26,240 has three eigenvectors, but not orthogonal. 264 00:14:26,240 --> 00:14:29,380 But if it happens to commute with its transpose, 265 00:14:29,380 --> 00:14:34,010 then, wonderfully, the eigenvectors are orthogonal. 266 00:14:34,010 --> 00:14:39,250 Now, do you see how symmetric matrices pass this test? 267 00:14:39,250 --> 00:14:40,340 Of course. 268 00:14:40,340 --> 00:14:43,790 If A transpose equals A, then both sides are A squared, 269 00:14:43,790 --> 00:14:45,640 we've got it. 270 00:14:45,640 --> 00:14:49,410 How do anti-symmetric matrices pass this test? 271 00:14:49,410 --> 00:14:54,060 If A transpose equals minus A, then we've 272 00:14:54,060 --> 00:14:58,220 got it again, because we've got minus A squared on both sides. 273 00:14:58,220 --> 00:15:00,050 So that's another group. 274 00:15:00,050 --> 00:15:03,090 And finally, let me ask you about our other favorite 275 00:15:03,090 --> 00:15:06,950 family, orthogonal matrices. 276 00:15:06,950 --> 00:15:11,730 Do orthogonal matrices pass this test, if A is a Q, 277 00:15:11,730 --> 00:15:14,990 do they pass the test for orthogonal eigenvectors. 278 00:15:14,990 --> 00:15:22,320 Well, if A is Q, an orthogonal matrix, what is Q transpose Q? 279 00:15:22,320 --> 00:15:23,020 It's I. 280 00:15:23,020 --> 00:15:25,310 And what is Q Q transpose? 281 00:15:25,310 --> 00:15:28,010 It's I, we're talking square matrices here. 282 00:15:28,010 --> 00:15:30,100 So yes, it passes the test. 283 00:15:30,100 --> 00:15:36,600 So the special cases are symmetric, anti-symmetric 284 00:15:36,600 --> 00:15:41,080 (I'll say skew-symmetric,) and orthogonal. 285 00:15:41,080 --> 00:15:44,280 Those are the three important special classes 286 00:15:44,280 --> 00:15:45,710 that are in this family. 287 00:15:45,710 --> 00:15:46,210 OK. 288 00:15:46,210 --> 00:15:52,860 That's like a comment that, could have been made back in, 289 00:15:52,860 --> 00:15:54,870 section six point four. 290 00:15:54,870 --> 00:16:04,390 OK, I can pursue the differential equations, also 291 00:16:04,390 --> 00:16:09,090 this question, didn't ask you to tell me, 292 00:16:09,090 --> 00:16:13,920 how would I find this matrix exponential, e to the At? 293 00:16:13,920 --> 00:16:15,050 So can I erase this? 294 00:16:15,050 --> 00:16:17,050 I'll just stay with this same... 295 00:16:17,050 --> 00:16:19,690 296 00:16:19,690 --> 00:16:23,770 how would I find e to the At? 297 00:16:23,770 --> 00:16:27,010 Because, how does that come in? 298 00:16:27,010 --> 00:16:30,140 That's the key matrix for a differential equation, 299 00:16:30,140 --> 00:16:32,460 because the solution is -- 300 00:16:32,460 --> 00:16:38,520 the solution is u(t) is e^(At) u(0). 301 00:16:38,520 --> 00:16:42,060 So this is like the fundamental matrix 302 00:16:42,060 --> 00:16:48,400 that multiplies the given function and gives the answer. 303 00:16:48,400 --> 00:16:53,630 And how would we compute it if we wanted that? 304 00:16:53,630 --> 00:16:56,830 We don't always have to find e to the At, because I 305 00:16:56,830 --> 00:17:00,500 can go directly to the answer without any e to the At-s, 306 00:17:00,500 --> 00:17:06,440 but hiding here is an e to the At, and how would I compute it? 307 00:17:06,440 --> 00:17:10,026 Well, if A is diagonalizable. 308 00:17:10,026 --> 00:17:12,589 309 00:17:12,589 --> 00:17:21,510 So I'm now going to put in my usual if A can be diagonalized 310 00:17:21,510 --> 00:17:25,240 (and everybody remember that there is an if there, 311 00:17:25,240 --> 00:17:28,820 because it might not have enough eigenvectors) 312 00:17:28,820 --> 00:17:33,330 this example does have enough, random matrices have enough. 313 00:17:33,330 --> 00:17:36,720 So if we can diagonalize, then we get a nice formula for this, 314 00:17:36,720 --> 00:17:40,050 because an S comes way out at the beginning, 315 00:17:40,050 --> 00:17:42,740 and S inverse comes way out at the end, 316 00:17:42,740 --> 00:17:47,450 and we only have to take the exponential of lambda. 317 00:17:47,450 --> 00:17:49,980 And that's just a diagonal matrix, 318 00:17:49,980 --> 00:17:53,780 so that's just e the lambda one t, 319 00:17:53,780 --> 00:18:00,030 these guys are showing up, now, in e to the lambda nt. 320 00:18:00,030 --> 00:18:01,070 OK? 321 00:18:01,070 --> 00:18:03,510 That's a really quick review of that formula. 322 00:18:03,510 --> 00:18:06,040 323 00:18:06,040 --> 00:18:08,780 It's something we can compute it quickly 324 00:18:08,780 --> 00:18:11,250 if we have done the S and lambda part. 325 00:18:11,250 --> 00:18:13,770 326 00:18:13,770 --> 00:18:15,470 If we know S and lambda, then it's 327 00:18:15,470 --> 00:18:17,440 not hard to take that step. 328 00:18:17,440 --> 00:18:20,770 OK, that's some comments on differential equations. 329 00:18:20,770 --> 00:18:28,330 I would like to go on to a next question that I started here. 330 00:18:28,330 --> 00:18:33,410 And it's, got several parts, and I can just read it out. 331 00:18:33,410 --> 00:18:37,320 What we're given is a three-by-three matrix, 332 00:18:37,320 --> 00:18:41,900 and we're told its eigenvalues, except one of these 333 00:18:41,900 --> 00:18:47,700 is, like, we don't know, and we're told the eigenvectors. 334 00:18:47,700 --> 00:18:50,500 And I want to ask you about the matrix. 335 00:18:50,500 --> 00:18:51,190 OK. 336 00:18:51,190 --> 00:18:54,530 So, first question. 337 00:18:54,530 --> 00:18:56,430 Is the matrix diagonalizable? 338 00:18:56,430 --> 00:18:59,000 339 00:18:59,000 --> 00:19:03,030 And I really mean for which c, because I 340 00:19:03,030 --> 00:19:06,850 don't know c, so my questions will all be, 341 00:19:06,850 --> 00:19:12,340 for which is there a condition on c, does one c work. 342 00:19:12,340 --> 00:19:17,270 But your answer should tell me all the c-s that work. 343 00:19:17,270 --> 00:19:21,390 I'm not asking for you to tell me, well, c equal four, 344 00:19:21,390 --> 00:19:22,660 yes, that checks out. 345 00:19:22,660 --> 00:19:27,927 I want to know all the c-s that make it diagonalizable. 346 00:19:27,927 --> 00:19:34,950 347 00:19:34,950 --> 00:19:36,640 OK? 348 00:19:36,640 --> 00:19:39,440 What's the real on diagonalizable? 349 00:19:39,440 --> 00:19:42,212 We need enough eigenvectors, right? 350 00:19:42,212 --> 00:19:43,920 We don't care what those eigenvalues are, 351 00:19:43,920 --> 00:19:46,710 it's eigenvectors that count for diagonalizable, 352 00:19:46,710 --> 00:19:49,220 and we need three independent ones, 353 00:19:49,220 --> 00:19:52,340 and are those three guys independent? 354 00:19:52,340 --> 00:19:53,580 Yes. 355 00:19:53,580 --> 00:19:56,300 Actually, let's look at them for a moment. 356 00:19:56,300 --> 00:19:59,921 What do you see about those three vectors right away? 357 00:19:59,921 --> 00:20:01,170 They're more than independent. 358 00:20:01,170 --> 00:20:04,380 359 00:20:04,380 --> 00:20:09,730 Can you see why those three got chosen? 360 00:20:09,730 --> 00:20:15,780 Because it will come up in the next part, they're orthogonal. 361 00:20:15,780 --> 00:20:17,920 Those eigenvectors are orthogonal. 362 00:20:17,920 --> 00:20:19,710 They're certainly independent. 363 00:20:19,710 --> 00:20:28,220 So the answer to diagonalizable is, yes, all c, all c. 364 00:20:28,220 --> 00:20:30,760 Doesn't matter. c could be a repeated guy, 365 00:20:30,760 --> 00:20:32,360 but we've got enough eigenvectors, 366 00:20:32,360 --> 00:20:33,930 so that's what we care about. 367 00:20:33,930 --> 00:20:36,400 OK, second question. 368 00:20:36,400 --> 00:20:38,365 For which values of c is it symmetric? 369 00:20:38,365 --> 00:20:40,960 370 00:20:40,960 --> 00:20:46,000 OK, what's the answer to that one? 371 00:20:46,000 --> 00:20:48,650 372 00:20:48,650 --> 00:20:53,420 If we know the same setup if we know that much about it, 373 00:20:53,420 --> 00:20:55,330 we know those eigenvectors, and we've 374 00:20:55,330 --> 00:21:02,850 noticed they're orthogonal, then which c-s will work? 375 00:21:02,850 --> 00:21:07,800 So the eigenvalues of that symmetric matrix have to be 376 00:21:07,800 --> 00:21:08,490 real. 377 00:21:08,490 --> 00:21:11,780 So all real c. 378 00:21:11,780 --> 00:21:17,300 If c was i, the matrix wouldn't have been symmetric. 379 00:21:17,300 --> 00:21:24,040 But if c is a real number, then we've got real eigenvalues, 380 00:21:24,040 --> 00:21:25,920 we've got orthogonal eigenvectors, 381 00:21:25,920 --> 00:21:27,400 that matrix is symmetric. 382 00:21:27,400 --> 00:21:28,790 OK, positive definite. 383 00:21:28,790 --> 00:21:40,630 OK, now this is a sub-case of symmetric, 384 00:21:40,630 --> 00:21:45,900 so we need c to be real, so we've got a symmetric matrix, 385 00:21:45,900 --> 00:21:50,360 but we also want the thing to be positive definite. 386 00:21:50,360 --> 00:21:52,340 Now, we're looking at eigenvalues, 387 00:21:52,340 --> 00:21:54,740 we've got a lot of tests for positive definite, 388 00:21:54,740 --> 00:21:57,250 but eigenvalues, if we know them, 389 00:21:57,250 --> 00:22:01,100 is certainly a good, quick, clean test. 390 00:22:01,100 --> 00:22:05,570 Could this matrix be positive definite? 391 00:22:05,570 --> 00:22:06,640 No. 392 00:22:06,640 --> 00:22:10,180 No, because it's got an eigenvalue zero. 393 00:22:10,180 --> 00:22:12,640 It could be positive semi-definite, 394 00:22:12,640 --> 00:22:15,900 you know, like consolation prize, 395 00:22:15,900 --> 00:22:19,320 if c was greater or equal to zero, 396 00:22:19,320 --> 00:22:21,680 it would be positive semi-definite. 397 00:22:21,680 --> 00:22:25,520 But it's not, no. 398 00:22:25,520 --> 00:22:30,670 Semi-definite, if I put that comment in, semi-definite, 399 00:22:30,670 --> 00:22:35,010 that the condition would be c greater or equal to zero. 400 00:22:35,010 --> 00:22:36,300 That would be all right. 401 00:22:36,300 --> 00:22:37,220 OK. 402 00:22:37,220 --> 00:22:38,330 Next part. 403 00:22:38,330 --> 00:22:39,675 Is it a Markov matrix? 404 00:22:39,675 --> 00:22:44,260 405 00:22:44,260 --> 00:22:44,870 Hm. 406 00:22:44,870 --> 00:22:50,440 Could this matrix be, if I choose the number c correctly, 407 00:22:50,440 --> 00:22:52,040 a Markov matrix? 408 00:22:52,040 --> 00:22:58,750 409 00:22:58,750 --> 00:23:02,700 Well, what do we know about Markov matrices? 410 00:23:02,700 --> 00:23:05,320 Mainly, we know something about their eigenvalues. 411 00:23:05,320 --> 00:23:10,500 One eigenvalue is always one, and the other eigenvalues 412 00:23:10,500 --> 00:23:13,430 are smaller. 413 00:23:13,430 --> 00:23:14,690 Not larger. 414 00:23:14,690 --> 00:23:17,380 So an eigenvalue two can't happen. 415 00:23:17,380 --> 00:23:22,000 So the answer is, no, not a ma- that's never a Markov matrix. 416 00:23:22,000 --> 00:23:22,750 OK? 417 00:23:22,750 --> 00:23:29,970 And finally, could one half of A be a projection matrix? 418 00:23:29,970 --> 00:23:32,820 So could it- could this -- eh-eh could this be twice 419 00:23:32,820 --> 00:23:33,890 a projection matrix? 420 00:23:33,890 --> 00:23:35,840 So let me write it this way. 421 00:23:35,840 --> 00:23:39,530 Could A over two be a projection matrix? 422 00:23:39,530 --> 00:23:44,820 423 00:23:44,820 --> 00:23:46,820 OK, what are projection matrices? 424 00:23:46,820 --> 00:23:48,650 They're real. 425 00:23:48,650 --> 00:23:53,160 I mean, th- they're symmetric, so their eigenvalues are real. 426 00:23:53,160 --> 00:23:56,680 But more than that, we know what those eigenvalues have to be. 427 00:23:56,680 --> 00:24:01,150 What do the eigenvalues of a projection matrix have to be? 428 00:24:01,150 --> 00:24:05,670 See, that any nice matrix we've got 429 00:24:05,670 --> 00:24:08,490 an idea about its eigenvalues. 430 00:24:08,490 --> 00:24:12,980 So the eigenvalues of projection matrices are zero and 431 00:24:12,980 --> 00:24:13,970 one. 432 00:24:13,970 --> 00:24:16,710 Zero and one, only. 433 00:24:16,710 --> 00:24:21,510 Because P squared equals P, let me call this matrix P, 434 00:24:21,510 --> 00:24:26,510 so P squared equals P, so lambda squared equals lambda, 435 00:24:26,510 --> 00:24:30,570 because eigenvalues of P squared are lambda squared, 436 00:24:30,570 --> 00:24:37,520 and we must have that, so lambda equals zero or one. 437 00:24:37,520 --> 00:24:38,140 OK. 438 00:24:38,140 --> 00:24:42,060 Now what value of c will work there? 439 00:24:42,060 --> 00:24:48,250 So, then, there are some value that will work, 440 00:24:48,250 --> 00:24:50,300 and what will work? 441 00:24:50,300 --> 00:24:56,340 c equals zero will work, or what else will work? 442 00:24:56,340 --> 00:24:59,310 443 00:24:59,310 --> 00:25:02,690 c equal to two. 444 00:25:02,690 --> 00:25:06,260 Because if c is two, then when we divide by two, 445 00:25:06,260 --> 00:25:09,880 this Eigenvalue of two will drop to one, 446 00:25:09,880 --> 00:25:13,110 and so will the other one, so, or c equal to two. 447 00:25:13,110 --> 00:25:15,640 OK, those are the guys that will work, 448 00:25:15,640 --> 00:25:21,110 and it was the fact that those eigenvectors were orthogonal, 449 00:25:21,110 --> 00:25:23,780 the fact that those eigenvectors were orthogonal 450 00:25:23,780 --> 00:25:26,170 carried us a lot of the way, here. 451 00:25:26,170 --> 00:25:29,400 If they weren't orthogonal, then symmetric would have been dead, 452 00:25:29,400 --> 00:25:31,420 positive definite would have been dead, 453 00:25:31,420 --> 00:25:33,150 projection would have been dead. 454 00:25:33,150 --> 00:25:37,090 But those eigenvectors were orthogonal, 455 00:25:37,090 --> 00:25:40,180 so it came down to the eigenvalues. 456 00:25:40,180 --> 00:25:45,270 OK, that was like a chance to review a lot of this chapter. 457 00:25:45,270 --> 00:25:50,790 458 00:25:50,790 --> 00:25:56,030 Shall I jump to the singular value decomposition, 459 00:25:56,030 --> 00:26:04,140 then, as the third, topic for, for the review? 460 00:26:04,140 --> 00:26:06,080 OK, so I'm going to. jump to this. 461 00:26:06,080 --> 00:26:06,580 OK. 462 00:26:06,580 --> 00:26:13,950 463 00:26:13,950 --> 00:26:16,990 So this is the singular value decomposition, 464 00:26:16,990 --> 00:26:21,070 known to everybody as the SVD. 465 00:26:21,070 --> 00:26:27,720 And that's a factorization of A into orthogonal times 466 00:26:27,720 --> 00:26:33,830 diagonal times orthogonal. 467 00:26:33,830 --> 00:26:41,030 And we always call those U and sigma and V transpose. 468 00:26:41,030 --> 00:26:42,300 OK. 469 00:26:42,300 --> 00:26:46,660 And the key to that -- 470 00:26:46,660 --> 00:26:51,170 this is for every matrix, every A, every A. 471 00:26:51,170 --> 00:26:54,110 Rectangular, doesn't matter, whatever, 472 00:26:54,110 --> 00:26:56,740 has this decomposition. 473 00:26:56,740 --> 00:26:59,070 So it's really important. 474 00:26:59,070 --> 00:27:04,930 And the key to it is to look at things like A transpose A. 475 00:27:04,930 --> 00:27:07,360 Can we remember what happens with A transpose A? 476 00:27:07,360 --> 00:27:11,300 If I just transpose that I get V sigma transpose U 477 00:27:11,300 --> 00:27:15,420 transpose, that's multiplying A, which is U, 478 00:27:15,420 --> 00:27:24,850 sigma V transpose, and the result is V on the outside, 479 00:27:24,850 --> 00:27:27,990 s- U transpose U is the identity, 480 00:27:27,990 --> 00:27:30,930 because it's an orthogonal matrix. 481 00:27:30,930 --> 00:27:34,550 So I'm just left with sigma transpose sigma 482 00:27:34,550 --> 00:27:39,340 in the middle, that's a diagonal, possibly 483 00:27:39,340 --> 00:27:42,960 rectangular diagonal by its transpose, so the result, 484 00:27:42,960 --> 00:27:46,036 this is orthogonal, diagonal, orthogonal. 485 00:27:46,036 --> 00:27:49,840 486 00:27:49,840 --> 00:27:55,620 So, I guess, actually, this is the SVD for A transpose A. 487 00:27:55,620 --> 00:27:59,930 Here I see orthogonal, diagonal, and orthogonal. 488 00:27:59,930 --> 00:28:00,440 Great. 489 00:28:00,440 --> 00:28:07,000 But a little more is happening. 490 00:28:07,000 --> 00:28:09,580 For A transpose A, the difference 491 00:28:09,580 --> 00:28:13,690 is, the orthogonal guys are the same. 492 00:28:13,690 --> 00:28:15,640 It's V and V transpose. 493 00:28:15,640 --> 00:28:17,290 What I seeing here? 494 00:28:17,290 --> 00:28:21,950 I'm seeing the factorization for a symmetric matrix. 495 00:28:21,950 --> 00:28:23,220 This thing is symmetric. 496 00:28:23,220 --> 00:28:26,790 497 00:28:26,790 --> 00:28:30,590 So in a symmetric case, U is the same as V. 498 00:28:30,590 --> 00:28:33,210 U is the same as V for this symmetric matrix, 499 00:28:33,210 --> 00:28:34,990 and, of course, we see it happening. 500 00:28:34,990 --> 00:28:35,540 OK. 501 00:28:35,540 --> 00:28:39,760 So that tells us, right away, what V is. 502 00:28:39,760 --> 00:28:49,650 V is the eigenvector matrix for A transpose A. 503 00:28:49,650 --> 00:28:50,490 OK. 504 00:28:50,490 --> 00:28:57,070 Now, if you were here when I lectured about this topic, when 505 00:28:57,070 --> 00:29:00,600 I gave the topic on singular value decompositions, 506 00:29:00,600 --> 00:29:03,180 you'll remember that I got into trouble. 507 00:29:03,180 --> 00:29:06,150 508 00:29:06,150 --> 00:29:09,860 I'm sorry to remember that myself, but it happened. 509 00:29:09,860 --> 00:29:10,550 OK. 510 00:29:10,550 --> 00:29:13,680 How did it happen? 511 00:29:13,680 --> 00:29:16,900 I was in great shape for a while, cruising along. 512 00:29:16,900 --> 00:29:20,120 So I found the eigenvectors for A transpose A. 513 00:29:20,120 --> 00:29:21,870 Good. 514 00:29:21,870 --> 00:29:24,800 I found the singular values, what were they? 515 00:29:24,800 --> 00:29:26,520 What were the singular values? 516 00:29:26,520 --> 00:29:32,720 The singular value number i, or -- 517 00:29:32,720 --> 00:29:36,890 these are the guys in sigma -- 518 00:29:36,890 --> 00:29:39,770 this is diagonal with the number sigma in it. 519 00:29:39,770 --> 00:29:42,910 This diagonal is sigma one, sigma two, 520 00:29:42,910 --> 00:29:46,090 up to the rank, sigma r, those are the non-zero ones. 521 00:29:46,090 --> 00:29:48,830 522 00:29:48,830 --> 00:29:51,100 So I found those, and what are they? 523 00:29:51,100 --> 00:29:53,130 Remind me about that? 524 00:29:53,130 --> 00:29:59,630 Well, here, I'm seeing them squared, so their squares are 525 00:29:59,630 --> 00:30:03,470 the eigenvalues of A transpose A. 526 00:30:03,470 --> 00:30:04,760 Good. 527 00:30:04,760 --> 00:30:09,160 So I just take the square root, if I want the eigenvalues of A 528 00:30:09,160 --> 00:30:10,020 transpose -- 529 00:30:10,020 --> 00:30:11,990 If I want the sigmas and I know these, 530 00:30:11,990 --> 00:30:14,540 I take the square root, the positive square root. 531 00:30:14,540 --> 00:30:16,730 OK. 532 00:30:16,730 --> 00:30:20,610 Where did I run into trouble? 533 00:30:20,610 --> 00:30:25,220 Well, then, my final step was to find U. 534 00:30:25,220 --> 00:30:28,270 And I didn't read the book. 535 00:30:28,270 --> 00:30:35,480 So, I did something that was practically right, but -- 536 00:30:35,480 --> 00:30:38,880 well, I guess practically right is not quite the same. 537 00:30:38,880 --> 00:30:44,650 OK, so I thought, OK, I'll look at A A transpose. 538 00:30:44,650 --> 00:30:47,420 What happened when I looked at A A transpose? 539 00:30:47,420 --> 00:30:51,070 Let me just put it here, and then I can feel it. 540 00:30:51,070 --> 00:30:53,620 OK, so here's A A transpose. 541 00:30:53,620 --> 00:30:57,120 542 00:30:57,120 --> 00:31:01,050 So that's U sigma V transpose, that's A, 543 00:31:01,050 --> 00:31:05,240 and then the transpose is V sigma transpose, 544 00:31:05,240 --> 00:31:06,300 U sigma transpose. 545 00:31:06,300 --> 00:31:07,930 Fine. 546 00:31:07,930 --> 00:31:10,610 And then, in the middle is the identity again, 547 00:31:10,610 --> 00:31:12,570 so it looks great. 548 00:31:12,570 --> 00:31:17,050 U sigma sigma transpose, U transpose. 549 00:31:17,050 --> 00:31:18,760 Fine. 550 00:31:18,760 --> 00:31:26,570 All good, and now these columns of U 551 00:31:26,570 --> 00:31:29,900 are the eigenvectors, that's U is the eigenvector 552 00:31:29,900 --> 00:31:33,120 matrix for this guy. 553 00:31:33,120 --> 00:31:36,960 That was correct, so I did that fine. 554 00:31:36,960 --> 00:31:38,600 Where did something go wrong? 555 00:31:38,600 --> 00:31:40,820 A sign went wrong. 556 00:31:40,820 --> 00:31:44,570 A sign went wrong because -- and now -- now I see, actually, 557 00:31:44,570 --> 00:31:49,140 somebody told me right after class, 558 00:31:49,140 --> 00:31:53,910 we can't tell from this description which sign to give 559 00:31:53,910 --> 00:31:55,200 the eigenvectors. 560 00:31:55,200 --> 00:32:00,570 If these are the eigenvectors of this matrix, 561 00:32:00,570 --> 00:32:02,660 well, if you give me an eigenvector 562 00:32:02,660 --> 00:32:04,790 and I change all its signs, we've 563 00:32:04,790 --> 00:32:06,920 still got another eigenvector. 564 00:32:06,920 --> 00:32:08,970 So what I wasn't able to determine 565 00:32:08,970 --> 00:32:13,940 (and I had a fifty-fifty change and life let me down,) 566 00:32:13,940 --> 00:32:16,600 the signs I just happened to pick 567 00:32:16,600 --> 00:32:19,070 for the eigenvectors, one of them 568 00:32:19,070 --> 00:32:21,750 I should have reversed the sign. 569 00:32:21,750 --> 00:32:27,190 So, from this, I can't tell whether the eigenvector 570 00:32:27,190 --> 00:32:31,150 or its negative is the right one to use in there. 571 00:32:31,150 --> 00:32:34,750 So the right way to do it is to, having 572 00:32:34,750 --> 00:32:38,640 settled on the signs, the Vs also, I 573 00:32:38,640 --> 00:32:42,100 don't know which sign to choose, but I choose one. 574 00:32:42,100 --> 00:32:43,220 I choose one. 575 00:32:43,220 --> 00:32:50,290 And then, instead, I should have used 576 00:32:50,290 --> 00:32:53,950 the one that tells me what sign to choose, the rule 577 00:32:53,950 --> 00:33:02,330 that A times a V is sigma times the U. 578 00:33:02,330 --> 00:33:07,140 So, having decided on the V, I multiply by A, 579 00:33:07,140 --> 00:33:09,640 I'll notice the factor sigma coming out, 580 00:33:09,640 --> 00:33:11,520 and there will be a unit vector there, 581 00:33:11,520 --> 00:33:17,310 and I now know exactly what it is, 582 00:33:17,310 --> 00:33:20,380 and not only up to a change of sign. 583 00:33:20,380 --> 00:33:22,390 So that's the good and, of course, 584 00:33:22,390 --> 00:33:25,910 this is the main point about the SVD. 585 00:33:25,910 --> 00:33:28,210 That's the point that we've diagonalized, 586 00:33:28,210 --> 00:33:32,950 that's A times the matrix of Vs equals 587 00:33:32,950 --> 00:33:37,710 U times the diagonal matrix of sigmas. 588 00:33:37,710 --> 00:33:39,470 That's the same as that. 589 00:33:39,470 --> 00:33:39,970 OK. 590 00:33:39,970 --> 00:33:47,800 So that's, like, correcting the wrong sign 591 00:33:47,800 --> 00:33:50,000 from that earlier lecture. 592 00:33:50,000 --> 00:33:52,810 And that would complete that, so that's how you would compute 593 00:33:52,810 --> 00:33:54,040 the SVD. 594 00:33:54,040 --> 00:33:58,380 Now, on the quiz, I going to ask -- well, maybe on the final. 595 00:33:58,380 --> 00:34:01,010 So we've got quiz and final ahead. 596 00:34:01,010 --> 00:34:05,400 Sometimes, you might be asked to find the SVD if I give you 597 00:34:05,400 --> 00:34:10,870 the matrix -- let me come back, now, to the main board -- 598 00:34:10,870 --> 00:34:17,880 or, I might give you the pieces. 599 00:34:17,880 --> 00:34:21,810 And I might ask you something about the matrix. 600 00:34:21,810 --> 00:34:31,580 For example, suppose I ask you, oh, let's say, 601 00:34:31,580 --> 00:34:36,590 if I tell you what sigma is -- 602 00:34:36,590 --> 00:34:37,460 OK. 603 00:34:37,460 --> 00:34:39,230 Let's take one example. 604 00:34:39,230 --> 00:34:43,820 Suppose sigma is -- 605 00:34:43,820 --> 00:34:46,350 so all that's how we would compute them. 606 00:34:46,350 --> 00:34:48,070 But now, suppose I give you these. 607 00:34:48,070 --> 00:34:52,320 Suppose I give you sigma is, say, three two. 608 00:34:52,320 --> 00:34:57,130 609 00:34:57,130 --> 00:35:02,110 And I tell you that U has a couple of columns, 610 00:35:02,110 --> 00:35:04,580 and V has a couple of columns. 611 00:35:04,580 --> 00:35:07,910 612 00:35:07,910 --> 00:35:10,330 OK. 613 00:35:10,330 --> 00:35:12,600 Those are orthogonal columns, of course, 614 00:35:12,600 --> 00:35:14,630 because U and V are orthogonal. 615 00:35:14,630 --> 00:35:16,120 I'm just sort of, like, getting you 616 00:35:16,120 --> 00:35:19,440 to think about the SVD, because we only had that one 617 00:35:19,440 --> 00:35:22,570 lecture about it, and one homework, 618 00:35:22,570 --> 00:35:28,460 and, what kind of a matrix have I got here? 619 00:35:28,460 --> 00:35:31,970 What do I know about this matrix? 620 00:35:31,970 --> 00:35:35,540 All I really know right now is that its singular values, 621 00:35:35,540 --> 00:35:39,390 those sigmas are three and two, and the only thing 622 00:35:39,390 --> 00:35:43,190 interesting that I can see in that is that they're not zero. 623 00:35:43,190 --> 00:35:48,390 I know that this matrix is non-singular, right? 624 00:35:48,390 --> 00:35:51,570 That's invertible, I don't have any zero eigenvalues, 625 00:35:51,570 --> 00:35:54,710 and zero singular values, that's invertible, 626 00:35:54,710 --> 00:36:02,290 there's a typical SVD for a nice two-by-two non-singular 627 00:36:02,290 --> 00:36:04,990 invertible good matrix. 628 00:36:04,990 --> 00:36:07,480 If I actually gave you a matrix, then you'd 629 00:36:07,480 --> 00:36:10,390 have to find the Us and the Vs as we just spoke. 630 00:36:10,390 --> 00:36:12,000 But, there. 631 00:36:12,000 --> 00:36:16,400 Now, what if the two wasn't a two but it was -- 632 00:36:16,400 --> 00:36:18,520 well, let me make an extreme case, here -- 633 00:36:18,520 --> 00:36:20,050 suppose it was minus five. 634 00:36:20,050 --> 00:36:23,220 635 00:36:23,220 --> 00:36:24,810 That's wrong, right away. 636 00:36:24,810 --> 00:36:28,590 That's not a singular value decomposition, right? 637 00:36:28,590 --> 00:36:30,840 The singular values are not negative. 638 00:36:30,840 --> 00:36:36,011 So that's not a singular value decomposition, and forget it. 639 00:36:36,011 --> 00:36:36,510 OK. 640 00:36:36,510 --> 00:36:40,200 So let me ask you about that one. 641 00:36:40,200 --> 00:36:42,095 What can you tell me about that matrix? 642 00:36:42,095 --> 00:36:45,090 643 00:36:45,090 --> 00:36:47,340 It's singular, right? 644 00:36:47,340 --> 00:36:50,480 It's got a singular matrix there in the middle, 645 00:36:50,480 --> 00:36:56,110 and, let's see, so, OK, it's singular, 646 00:36:56,110 --> 00:37:01,870 maybe you can tell me, its rank? 647 00:37:01,870 --> 00:37:04,320 What's the rank of A? 648 00:37:04,320 --> 00:37:08,900 It's clearly -- somebody just say it -- 649 00:37:08,900 --> 00:37:09,920 one, thanks. 650 00:37:09,920 --> 00:37:15,160 The rank is one, so the null space, 651 00:37:15,160 --> 00:37:18,850 what's the dimension of the null space? 652 00:37:18,850 --> 00:37:19,890 One. 653 00:37:19,890 --> 00:37:20,390 Right? 654 00:37:20,390 --> 00:37:23,940 We've got a two-by-two matrix of rank one, 655 00:37:23,940 --> 00:37:26,820 so of all that stuff from the beginning of the course 656 00:37:26,820 --> 00:37:30,310 is still with us. 657 00:37:30,310 --> 00:37:32,790 The dimensions of those fundamental spaces 658 00:37:32,790 --> 00:37:36,950 is still central, and a basis for them. 659 00:37:36,950 --> 00:37:40,890 Now, can you tell me a vector that's in the null space? 660 00:37:40,890 --> 00:37:47,000 And then that will be my last point to make about the SVD. 661 00:37:47,000 --> 00:37:49,400 Can you tell me a vector that's in the null space? 662 00:37:49,400 --> 00:37:54,410 663 00:37:54,410 --> 00:38:00,820 So what would I multiply by and get zero, here? 664 00:38:00,820 --> 00:38:04,120 I think the answer is probably v2. 665 00:38:04,120 --> 00:38:07,820 I think probably v2 is in the null space, 666 00:38:07,820 --> 00:38:11,950 because I think that must be the eigenvector going 667 00:38:11,950 --> 00:38:14,800 with this zero eigenvalue. 668 00:38:14,800 --> 00:38:16,310 Yes. 669 00:38:16,310 --> 00:38:17,200 Have a look at that. 670 00:38:17,200 --> 00:38:21,390 And I could ask you the null space of A transpose. 671 00:38:21,390 --> 00:38:23,050 And I could ask you the column space. 672 00:38:23,050 --> 00:38:24,620 All that stuff. 673 00:38:24,620 --> 00:38:27,180 Everything is sitting there in the SVD. 674 00:38:27,180 --> 00:38:29,990 The SVD takes a little more time to compute, 675 00:38:29,990 --> 00:38:36,090 but it displays all the good stuff about a matrix. 676 00:38:36,090 --> 00:38:36,720 OK. 677 00:38:36,720 --> 00:38:39,680 Any question about the SVD? 678 00:38:39,680 --> 00:38:47,800 Let me keep going with further topics. 679 00:38:47,800 --> 00:38:48,990 Now, let's see. 680 00:38:48,990 --> 00:38:51,050 Similar matrices we've talked about, 681 00:38:51,050 --> 00:38:55,390 let me see if I've got another, -- 682 00:38:55,390 --> 00:38:57,620 OK. 683 00:38:57,620 --> 00:39:04,570 Here's a true false, so we can do that, easily. 684 00:39:04,570 --> 00:39:05,300 So. 685 00:39:05,300 --> 00:39:07,560 Question, A given. 686 00:39:07,560 --> 00:39:10,480 687 00:39:10,480 --> 00:39:17,840 A is symmetric and orthogonal. 688 00:39:17,840 --> 00:39:20,861 689 00:39:20,861 --> 00:39:21,360 OK. 690 00:39:21,360 --> 00:39:26,920 691 00:39:26,920 --> 00:39:29,870 So beautiful matrices like that don't come along every day. 692 00:39:29,870 --> 00:39:36,480 But what can we say first about its eigenvalues? 693 00:39:36,480 --> 00:39:38,680 Actually, of course. 694 00:39:38,680 --> 00:39:41,810 Here are our two most important classes of matrices, 695 00:39:41,810 --> 00:39:45,500 and we're looking at the intersection. 696 00:39:45,500 --> 00:39:48,470 So those really are neat matrices, 697 00:39:48,470 --> 00:39:50,920 and what can you tell me about what could 698 00:39:50,920 --> 00:39:52,670 the possible eigenvalues be? 699 00:39:52,670 --> 00:39:57,540 Eigenvalues can be what? 700 00:39:57,540 --> 00:39:59,330 What do I know about the eigenvalues 701 00:39:59,330 --> 00:40:01,280 of a symmetric matrix? 702 00:40:01,280 --> 00:40:04,690 Lambda is real. 703 00:40:04,690 --> 00:40:06,550 What do I know about the eigenvalues 704 00:40:06,550 --> 00:40:10,260 of an orthogonal matrix? 705 00:40:10,260 --> 00:40:12,370 Ha. 706 00:40:12,370 --> 00:40:13,200 Maybe nothing. 707 00:40:13,200 --> 00:40:15,420 But, no, that can't be. 708 00:40:15,420 --> 00:40:17,911 What do I know about the eigenvalues of an orthogonal 709 00:40:17,911 --> 00:40:18,410 matrix? 710 00:40:18,410 --> 00:40:19,565 Well, what feels right? 711 00:40:19,565 --> 00:40:22,330 712 00:40:22,330 --> 00:40:26,620 Basing mathematics on just a little gut instinct here, 713 00:40:26,620 --> 00:40:29,910 the eigenvalues of an orthogonal matrix 714 00:40:29,910 --> 00:40:33,800 ought to have magnitude one. 715 00:40:33,800 --> 00:40:36,280 Orthogonal matrices are like rotations, 716 00:40:36,280 --> 00:40:40,921 they're not changing the length, so orthogonal, the eigenvalues 717 00:40:40,921 --> 00:40:41,420 are one. 718 00:40:41,420 --> 00:40:50,070 Let me just show you why. 719 00:40:50,070 --> 00:40:53,920 So the matrix, can I call it Q for orthogonal 720 00:40:53,920 --> 00:40:55,810 Why? for the moment? 721 00:40:55,810 --> 00:40:58,760 If I look at Q x equal lambda x, how 722 00:40:58,760 --> 00:41:03,570 do I see that this thing has magnitude one? 723 00:41:03,570 --> 00:41:06,140 I take the length of both sides. 724 00:41:06,140 --> 00:41:08,930 This is taking lengths, taking lengths, 725 00:41:08,930 --> 00:41:13,520 this is whatever the magnitude is times the length of x. 726 00:41:13,520 --> 00:41:18,170 And what's the length of Q x if Q is an orthogonal matrix? 727 00:41:18,170 --> 00:41:20,910 This is something you should know. 728 00:41:20,910 --> 00:41:23,100 It's the same as the length of x. 729 00:41:23,100 --> 00:41:26,400 Orthogonal matrices don't change lengths. 730 00:41:26,400 --> 00:41:30,330 So lambda has to be one. 731 00:41:30,330 --> 00:41:31,090 Right. 732 00:41:31,090 --> 00:41:31,890 OK. 733 00:41:31,890 --> 00:41:34,440 That's worth committing to memory, 734 00:41:34,440 --> 00:41:36,930 that could show up again. 735 00:41:36,930 --> 00:41:37,430 OK. 736 00:41:37,430 --> 00:41:40,720 So what's the answer now to this question, 737 00:41:40,720 --> 00:41:42,940 what can the eigenvalues be? 738 00:41:42,940 --> 00:41:45,220 There's only two possibilities, and they 739 00:41:45,220 --> 00:41:55,050 are one and the other one, the other possibility 740 00:41:55,050 --> 00:42:00,770 is negative one, right, because these have the right magnitude, 741 00:42:00,770 --> 00:42:03,090 and they're real. 742 00:42:03,090 --> 00:42:04,070 OK. 743 00:42:04,070 --> 00:42:04,730 TK. 744 00:42:04,730 --> 00:42:07,090 true -- OK. 745 00:42:07,090 --> 00:42:08,260 True or false? 746 00:42:08,260 --> 00:42:12,220 A is sure to be positive definite. 747 00:42:12,220 --> 00:42:14,070 Well, this is a great matrix, but is it 748 00:42:14,070 --> 00:42:16,570 sure to be positive definite? 749 00:42:16,570 --> 00:42:17,090 No. 750 00:42:17,090 --> 00:42:19,380 If it could have an eigenvalue minus one, 751 00:42:19,380 --> 00:42:21,910 it wouldn't be positive definite. 752 00:42:21,910 --> 00:42:24,230 True or false, it has no repeated eigenvalues. 753 00:42:24,230 --> 00:42:27,820 754 00:42:27,820 --> 00:42:29,700 That's false, too. 755 00:42:29,700 --> 00:42:31,710 In fact, it's going to have repeated eigenvalues 756 00:42:31,710 --> 00:42:33,740 if it's as big as three by three, 757 00:42:33,740 --> 00:42:35,630 one of these c- one of these, at least, 758 00:42:35,630 --> 00:42:37,270 will have to get repeated. 759 00:42:37,270 --> 00:42:37,770 Sure. 760 00:42:37,770 --> 00:42:40,040 So it's got repeated eigenvalues, but, 761 00:42:40,040 --> 00:42:42,910 is it diagonalizable? 762 00:42:42,910 --> 00:42:45,210 It's got these many, many, repeated eigenvalues. 763 00:42:45,210 --> 00:42:46,900 If it's fifty by fifty, it's certainly 764 00:42:46,900 --> 00:42:48,790 got a lot of repetitions. 765 00:42:48,790 --> 00:42:51,370 Is it diagonalizable? 766 00:42:51,370 --> 00:42:52,110 Yes. 767 00:42:52,110 --> 00:42:55,500 All symmetric matrices, all orthogonal matrices 768 00:42:55,500 --> 00:42:57,090 can be diagonalized. 769 00:42:57,090 --> 00:43:02,620 And, in fact, the eigenvectors can even be chosen orthogonal. 770 00:43:02,620 --> 00:43:05,120 So it could be, sort of, like, diagonalized 771 00:43:05,120 --> 00:43:09,270 the best way with a Q, and not just any old S. 772 00:43:09,270 --> 00:43:09,840 OK. 773 00:43:09,840 --> 00:43:11,820 Is it non-singular? 774 00:43:11,820 --> 00:43:15,275 Is a symmetric orthogonal matrix non-singular? 775 00:43:15,275 --> 00:43:19,500 776 00:43:19,500 --> 00:43:21,790 Orthogonal matrices are always non-singular. 777 00:43:21,790 --> 00:43:22,900 Sure. 778 00:43:22,900 --> 00:43:26,910 And, obviously, we don't have any zero Eigenvalues. 779 00:43:26,910 --> 00:43:28,670 Is it sure to be diagonalizable? 780 00:43:28,670 --> 00:43:31,310 Yes. 781 00:43:31,310 --> 00:43:41,370 Now, here's a final step -- show that one-half of A plus I is A 782 00:43:41,370 --> 00:43:42,140 -- 783 00:43:42,140 --> 00:43:51,705 that is, prove one-half of A plus I is a projection matrix. 784 00:43:51,705 --> 00:43:58,880 785 00:43:58,880 --> 00:43:59,380 OK? 786 00:43:59,380 --> 00:44:04,080 787 00:44:04,080 --> 00:44:04,750 Let's see. 788 00:44:04,750 --> 00:44:05,510 What do I do? 789 00:44:05,510 --> 00:44:09,170 790 00:44:09,170 --> 00:44:11,610 I could see two ways to do this. 791 00:44:11,610 --> 00:44:14,560 I could check the properties of a projection matrix, which 792 00:44:14,560 --> 00:44:15,700 are what? 793 00:44:15,700 --> 00:44:18,520 A projection matrix is symmetric. 794 00:44:18,520 --> 00:44:21,790 Well, that's certainly symmetric, because A is. 795 00:44:21,790 --> 00:44:24,440 And what's the other property? 796 00:44:24,440 --> 00:44:26,230 I should square it, and hopefully 797 00:44:26,230 --> 00:44:27,900 get the same thing back. 798 00:44:27,900 --> 00:44:32,230 So can I do that, square and see if I get the same thing back? 799 00:44:32,230 --> 00:44:37,060 So if I square it, I'll get one-quarter of A squared 800 00:44:37,060 --> 00:44:40,640 plus two A plus I, right? 801 00:44:40,640 --> 00:44:48,430 And the question is, does that agree with p- the thing itself? 802 00:44:48,430 --> 00:44:53,480 One-half A plus I. 803 00:44:53,480 --> 00:44:53,980 Hm. 804 00:44:53,980 --> 00:44:57,240 805 00:44:57,240 --> 00:45:02,060 I guess I'd like to know something about A squared. 806 00:45:02,060 --> 00:45:03,260 What is A squared? 807 00:45:03,260 --> 00:45:05,090 That's our problem. 808 00:45:05,090 --> 00:45:06,260 What is A squared? 809 00:45:06,260 --> 00:45:11,560 810 00:45:11,560 --> 00:45:13,510 If A is symmetric and orthogonal, 811 00:45:13,510 --> 00:45:17,390 A is symmetric and orthogonal. 812 00:45:17,390 --> 00:45:22,060 813 00:45:22,060 --> 00:45:23,710 This is what we're given, right? 814 00:45:23,710 --> 00:45:27,990 It's symmetric, and it's orthogonal. 815 00:45:27,990 --> 00:45:31,350 So what's A squared? 816 00:45:31,350 --> 00:45:36,360 I. A squared is I, because A times A -- 817 00:45:36,360 --> 00:45:42,500 if A equals its own inverse, so A times A is the same as A 818 00:45:42,500 --> 00:45:46,150 times A inverse, which is I. 819 00:45:46,150 --> 00:45:52,320 So this A squared here is I. 820 00:45:52,320 --> 00:45:54,530 And now we've got it. 821 00:45:54,530 --> 00:45:57,810 We've got two identities over four, that's good, 822 00:45:57,810 --> 00:46:01,060 and we've got two As over four, that's good. 823 00:46:01,060 --> 00:46:01,670 OK. 824 00:46:01,670 --> 00:46:05,960 So it turned out to be a projection matrix safely. 825 00:46:05,960 --> 00:46:08,310 And we could also have said, well, 826 00:46:08,310 --> 00:46:11,230 what are the eigenvalues of this thing? 827 00:46:11,230 --> 00:46:14,870 What are the eigenvalues of a half A plus I? 828 00:46:14,870 --> 00:46:17,970 If the eigenvalues of A are one and minus one, 829 00:46:17,970 --> 00:46:21,840 what are the eigenvalues of A plus I? 830 00:46:21,840 --> 00:46:25,860 Just stay with it these last thirty seconds here. 831 00:46:25,860 --> 00:46:29,140 What if I know these eigenvalues of A, 832 00:46:29,140 --> 00:46:31,480 and I add the identity, the eigenvalues 833 00:46:31,480 --> 00:46:35,600 of A plus I are zero and two. 834 00:46:35,600 --> 00:46:39,480 And then when I divide by two, the eigenvalues are zero and 835 00:46:39,480 --> 00:46:40,020 one. 836 00:46:40,020 --> 00:46:43,130 So it's symmetric, it's got the right eigenvalues, 837 00:46:43,130 --> 00:46:45,070 it's a projection matrix. 838 00:46:45,070 --> 00:46:49,290 OK, you're seeing a lot of stuff about eigenvalues, 839 00:46:49,290 --> 00:46:54,460 and special matrices, and that's what the quiz is about. 840 00:46:54,460 --> 00:46:57,230 OK, so good luck on the quiz. 841 00:46:57,230 --> 00:47:05,147